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ABSTRACT 


We-"  propose' a  test  for  detecting  serial  dependence  among  multivariate 

m 

observations*  -Our  test  statistic  is  the  maximum  absolute  value  of  the  lag  1 
correlation  obtainable  from  a  linear  combination  of  the  observations.  He  ' 
express  the  statistic  in  terms  of  two  eigenvalues  and  then  obtain  the 
asymptotic  null  distribution.  Asymptotic  power  is  examined  for  sequences  of 
local  alternatives  in  a  multivariate  normal  autoregressive  process.  An 

explicit  expression  is  obtained  for  the  density  of  the  limit  distribution  in 

'  v 

the  bivariate  case.  WC  then  compare  power  with  the  likelihood  ratio 

statistic.  '  W  A  ,  A  . 
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SIGNIFICANCE  AND  EXPLANATION 


If  multivariate  observations  taken  at  adjacent  times  are  correlated  the 
quality  of  inferences,  based  on  an  independence  assumption,  can  be  seriously 
eroded.  After  illustrating  these  effects,  we  propose  a  new  test  for  detecting 
dependence  asong  adjacent  observations.  We  reduce  the  problem  to  one 
dimension  by  considering  linear  combinations  a^X^  +  •••  +  a^X^  of  the 
observations . 

Our  test  statistic  is  then  the  maximum,  over  all  linear  combinations,  of 
the  sample  auto-corr e lation .  We  determine  its  large  sample  null  distribution 
from  which  approximate  critical  values  can  be  obtained  numerically. 

Because  of  the  seriousness  of  departures  from  independence,  it  is 
important  to  have  procedures  for  detecting  dependence.  Our  statistic  provides 
one  alternative  way  of  quantifying  dependence  in  a  series  of  multivariate 
observations.  It  should  be  a  useful  addition  to  summary  descriptions  of 
multivariate  data  sets  and  serve  as  a  warning  when  multivariate  time  series 
methods  are  required. 
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A  LINEAR  COMBINATION  TEST'  FOR  DETECTING  SERIAL  CORRELATION  IN  MULTIVARIATE  SAMPLES 


Richard  A.  Johnson  and  Thore  Langeland 

1.  INTRODUCTION 

It  > s  well-known  that  the  presence  of  even  a  moderate  autocorrelation,  among 
univariate  obaervationsi  can  cause  serious  difficulties  for  procedures  based  on  an 
assumption  of  independence.  In  the  multivariate  case,  both  inferences  about  the  mean  £ , 
and  covariance  matrix,  Z,  can  be  severely  affected.  To  illustrate,  let  Xt  follow  the 
multivariate  AR(  1 )  model 

«t  -  *  -  -  a>  +  St 

where  the  and  independent  and  identically  distributed  with  E(£t)  «  0  and 

Cov(E. )  •  £  and  all  the  eigenvalues  of  ♦  are  between  -V  and  1.  As  a  consequence 
6 

of  the  ergodic  theorem 

X  jj,  s  -  irL-  (it  -  XHJ^  -  x>-  ^  . 


Also 


Covtn"1^ 


n 

l 

t-i 


*t>  *  '-1 


♦> 


♦  Z„(i  -  ♦*) 


-t 


and  /ii(X  -  £)  is  asymptotically  normal  with  this  limiting  covariance  matrix.  If  the 
underlying  process  has  *  -  dlk,  | |  <  *,  then  the  nominal  95%  confidence  ellipsoid 
{ji  i  n(X  -  £)'S-1(X  -  Ji)  <  xkt»05l)  has  limiting  coverage  probability 
P(X^  <  ~  ^  x^(.05)].  For  instance  if  ^  h*e  JAewnsion  k  »  5  and  d  *  .3  the 

coverage  probability  is  .690.  For  k  •  10  and  d  ■  .5  it  is  .193. 
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In  the  context  of  principal  component  analysis,  suppose  we  wish  to  analyze  f  £ , 

which  is  the  covariance  matrix  for  under  independence,  but  that  the  correlation 

structure  is  introduced  by  selecting  a  sampling  interval  that  is  too  short,  lhe  first 

principal  component  has  coefficient  vector  e^  where  £e£i  *  with  >  •*•  >  X^. 

However,  if  the  underlying  ore  ss  has  #  =  C  t  1  where  C  is  just  smaller  than  X  , 

£  P 


so  y  ■  is  incorrectly  identified  as  the  coefficient  vector  of  the  first  principal 
component . 

Numerous  tests  have  been  proposed  for  the  univariate  case.  Ligget  (1977),  Bartlett 
ard  Rajaiaksham  (1953)',  and  Chittu.i  (1974)  have  proposed  multivariate  extensions  of  the 
Bartlett  periodogram  test,  the  Quenoulle  test  and  Box  and  Pierce  test,  respectively. 


2.  A  LINEAR  COMBINATION  TEST 

Because  first  order  autocorrelation  is  most  common,  it  is  worthwhile  to  develop  a 
test  for  first  order  correlation  that  is  both  easy  to  apply  and  has  a  graphic 
interpretation.  We  reduce  the  problem  to  one  dimension  by  considering  linear  combinations 
t  •  1,2,  ...,T  and  selecting  g  to  maximize  the  lag  1  correlation 


f  (1) 
a 


where  the  sample  cross-covariance  matrix  of  lag  j  is 


•  .  T-j 

Ci  -  -  l  (X  -  XXX  -  X)’  for  j  -  0,1, ...,T  -  1  .  (2.1) 

3  .  T  t-1  ~t  ~  ~t+l 


Our  test  statistic  is  then  defined  as  the  maximum  attainable  lag  1  correlation, 

RL  *  sup  |  r  ( 1 )  |  . 
a*0  £ 


Setting  C#  •  2_1(C1  +  Cj),  ra(1)  can  be  expressed  in  terms  of  symmetric  matrices  as 

|a'C  a |  . 

rl  “ 8up -irc^r' “  max{lxil (2,2) 

a^o  ~  0~ 

where  <  X2  <  •••  <  A^  are  the  eigenvalues  of  Cg'^CgCg1^2  or  CQ,CS*  0118  P°int  ft 
difficulty  is  that  Cg  is  not  necessarily  non-negative  definite. 

,  Note  that  R^  has  the  properties 


-3 


xt  ♦  a^q 


(ii)  Rl  is  invariant  under 

where  A  is  non-singular  and  a  orthogonal. 

A  » 

A  plot  of  a' (X^  -  X)  versus  a' (Xfc+1  -  X)  displays  the  concentrated  correlation 
estimated  by  RL- 

We  now  indicate  the  steps  leading  to  the  asymptotic  null  distribution  for  R^ 

leaving  the  more  technical  algebraic  steps  until  Section  5.  We  say  that  the  It  *  k 

matrix  B  is  N  .(0,Jl  if1)  if  tr(A'B)  is  N(0,trAfA‘$  ')  for  every  It  *  k  matrix 
It 

A.  Mann  and  Wald  (1943)  showed  that 

*  N  (04  a  j:-1) 

so  T1^2Cq1/,2CbCq1^*  — — *  S  where,  under  the  null  hypothesis,  S  has  pdf 

f(S,  -  (2.)-*C(k+1,/42k(,t-1,/4etr(-iSS,  ,  ,2.3, 

with  respect  to  k|k  +  1)/2  dimensional  Lebesgue  measure. 

Hsu  (1939)  encountered  the  same  asymptotic  distribution  while  studying  a  normal 
theory  one-way  MANOVA  problem.  He  established  that,  if  S  is  distributed  as  (2.3),  the 
distribution  of  its  eigenvalues  Xj  <  •••  <  X^  has  pdf 

k  2 

...  k  k  -H‘/2 

9,x  ,x  ,...,x  )  «  n  r,i/2,]-’  n  (X.  -  x.j  •  a  1-1  1  .  ,2.4, 

*  i-1  i<3  3  1 

Since  tV2Rl  a  continuous  function  of  T^cJJ^CgC JJ1/2, 

/t  Rj^  ' *  maxTjT,!  ,Xk)  .  (2.5, 

r 

For  k  •  2,  the  limit  distribution  is  easy  to  evaluate 

P(T1/2RL  <  xj  ♦  P(-x  <  A,  <  Aj  <  x]  -  H  /  ue”'*  ' 6(u)du  -  F(x)  .  (2.6, 

.  -X 

It  is  considerably  wore  difficult  to  prassnt  expressions  for  the  general  case.  Set 


// 


u  2 

3 j ( t )  -  /  u3e'u  ^du,  j  *  0,1,2, ... ,k  , 


x  i  t2/2 

G.  ,(x)  -  /  G.(t)t  e  7  dt,  0  <  j,i  <  k 
3,i  -x  5 


(2.7) 


(2.8) 


where  it  can  be  shown  (see  Mehta  (1960),  p.  399,  e-^n.  (13)) 

G  (x)  -  (-1)t+3G(x>  .  (2.9) 

3,*  i»3 

In  Section  5,,  we  establish 

Theorem.  2. 1 .  F^r  V.  even,  the  asymptotic  cdf  of  the  LCT  statistic  T1/,2RL,  under 
the  null  hypothesis  of  independence,  is 


F(x)  -  (  n  T(i/2) )-1det( {G  ,(x)>) 
i-1  3,1 


for  j  «  0,2,4, ...,k  -  2  and  1  -  1,3,5, „..,k  -  1,  where  G^  t(x)  is  defined  in  (2.8). 

Theorem  2.2  .  For  k  odd,  the  asymptotic  cdf  of  the  LCT  statistic  T1/,2rl,  under 
the  null  hypothesis  of  independence,  is 


,V2 


-1 


<k7t)/2 


5i  ' 


i-1 

j-0 

J  J 

i  (2.7), 

G0, 1(x} 

G0,3(x> 

•••  Go,k-2*x* 

G2,i<x) 

G2,3<x> 

•••  ®2,k-2*** 

- ♦ 

e 

G2j-2, 1(x) 

G2j-2,3(x> 

•••  G2j-2,k-2(x) 

<?2js-2,  1<*> 

g23+2,3(x> 

•••  G2j+2,k-2(x) 

°k-1, 1(x) 

ck-1,3<x> 

...  Gk-1,k-2*x* 

and  G. 

J 

£(x)  is  defined 

in  (2.8). 
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tt 


A  table  of  1-st,  5-th  and  10-th  percentiles,  for  It  »  2(1)20  were  calculated  using 


double  precision  arithmetic  (see  Lanqeland  (1980)) 


3.  SC  1  COMPETING  TESTS  AND  POWER  CONSIDERATIONS 

Most  tests  for  independence  are  motivated  from  consideration  of  autoregressive 
alternatives •  Let 

Xt  -  U  ’  *(Xt_1  -  «>  +  JEt 

for  t  “  1,2,  ...,T.  The  hypothesis  of  independence  is  the.i 

H  :  t1  =■  0  .  (3.1) 

A  natural  test  statistic  to  use  is 

SL  «-(*-{  Oc  +  k  +  1 )  ]  log[  jc0  -  ♦CQi,|/|C()|)  ,  (3.2) 

where  N  ■*  T  -  1  -  1  and  *  «  C 1  •  If  the  ate  ■ulti'»ariate  normal, 

then  the  test  statistic  in  (3.2)  has  the  same  asymptotic  distribution  as  the  logarithm  of 

the  likelihood  ratio  test  statistic.  See  Hannan  (1970,  pages  338-341). 

Theorem  3.1.  Under  the  null  hypothesis  of  independence  (3.1),  the  asymptotic 

2 

distribution  of  the  test  statistic  (3.2)  is  a  X  --distribution. 

k 

In  order  to  obtain  an  indication  of  asymptotic  power,  we  introduce  the  normal  theory 

AR(1)  model  (3.1)  where  the  ^  are  independent  N(g, fe ) .  Let  be  a  sequence  of 

1/2 

alternatives  to  independence,  where  T  '  ♦  ♦  H,  and  let  P  ,  denote  the  distribution 

T 

of  ^ , . . .  <  Let  PT  be  the  distribution  of  ^,... tJ£p  under  independence. 

Theorem  3.2.  Under 


tr(^1T1/2*TTV2C1]  -  \  trlfeT1/2*TC0TV2*y  ♦  op  (1) 


f(x)  -  4e"(X+n)/2  l  t^2?3  I  C/^)1  X211*1*11 

;-o  (jo2  i-o  11  r(lijl) 

•  J1  (1  -  u)23+1u2i6-x2!u2+^-u)2Jdu 
0 

for  x  >  0,  where  n  «  <w1  +  m3>2/2,  X  -  ((t^  -  u3)2  +  4y2]/2  and 

0  0  0 

1/2  1/2  0 

0  0  1 

It  also  follows  directly  that  ar®  each  jointly  normal  under  {p^}.  From  the 

continguity,  we  then  obtain 

Theorem  3.3.  Under  {PT  t  },  the  asymptotic  distribution  of  ^  is  non-central 

2  ^ 

X  j  with  nonce.itrality  parameter  tr[t  HI  H']. 

k  £  £ 

It  is  well-known  that  the  likelihood  ra.io  test  has  several  large  sample  optimal 
properties.  However,  a  calculation  of  asymptotic  power  in  Table  3.1  with  k  •  2,  (c  •  1 
shows  that  the  linear  combination  test  has  higher  power  than  the  others  when 
TV2*t  ♦  diag(hivn). 


-8- 


X 


«L 

SL 

.0513 

.0505 

.0849 

.0627 

.1796 

.1055 

.4666 

.320  1 

.7714 

.6635 

.9952 

.9894 

4. _ EXAMPLE 


We  consider  sons  data  reported  by  Simon  (see  tXinean  (1959),  pages  626-630)  consisting 
of  burning  times  of  30  fuses  as  recorded  by  three  observers.  Since  there  is  one  missing 
observation  for  the  second  observer,  we  first  confine  ourselves  to  the  data  given  by 
observers  one  and  three.  Let  m  txi,t'x2,t^ *'  ^  ”  1,2,. ..,30  denote  the 
observations.  The  plot  of  X^  ^ t  versus  t+|  for  i  ■  1  is  given  below  in  Figure 
4.1.  The  plot  for  i  -  2  is  similar.  Neither  exhibits  clear  signs  of  first  order  serial 
dependence.  The  LCT  statistic  /30  -  2.40  and  it  is  significant  at  the  10  -mrcent 

level.  The  value  of  the  corresponding  eigenvector  is  a  »  (1.0, -.99)'.  The  plot  of 
a 1 X^  versus  a'Xt+1  given  in  Figure  4.2  gives  an  indication  of  serial  dependence  in  the 
two  series  cf  data.  If  the  missing  observation  is  estimated,  the  evidence  for  dependence 
with  three  observers  is  much  stronger.  The  statistic  becomes  significant  at  the  3%  level. 


r1 

<  ’ 

if 

V 


5.  DERIVATIION  OF  LIMITING  NULL  DISTRIBUTION 

Hi*  asymptotic  cdf  of  T1/,2RL  is  given  by 

F(x)  «  P(-x  <  A1  <  A <  x)  -■/•••  /  g(A,, 

Q(-x,x.) 


*VdX1  “•  Xk 


(5,1) 


where  g(«)  is  defined  in  (2.4)  and  Q(a,b)  «  {a  <  <  Aj 

1  1 

A,  A, 


II  '  ( A  ,  -  A.)  det 

KKjOt  3 


aT1  a*-’ 


<  A^  <  b} .  Since 


,k-1 


(the  Vandermonde  determinant),  (5.1),  can  be  rewritten  as 


i 


V 

V 

V 

I 


i 


•r. 

V, 


/  •••  /  C  det 
Q(-x,x) 


“A2/2 


“A2/2 


-A^/2 


X2* 


■Aj/2 


v-1  "X1/2 
L  Xi  e 


,k-1  'X2/2 

a2  . 


•  dA^  •  dA2  •••  dA^ 


-Xk/2 


-Xk/2 


V 


V-,  -v* 


(5.2) 


where  c*  -  (2k/2  l  r( j/2J J r  . 

j-1  . 

In  order  to  obtain  an  explicit  expression  for  the  densities  we  need  some  additional 

concepts  and  lemmas  (see  Aitken  (1939),  pages  50  and  111). 

The  signature  function  S(x^,X2,>>*,xk).  is  defined  as 

E(x1,x2,  ...,xk)  •  It  sign<Xj  -  Xj)  (5.3) 

1<i< j<k 


for  x  -  (x1,x2,...,xk)’  «  H?*,  E(x^ ,xj, ... , x^)  •  0  if  x^  •  Xj  for 
i,j  “  1,2, ...,k,  and  B(x1)  »  1  for  all  x1  8  R. 


I  t  J, 
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Let  k  -  2a  and  a  “  1,2,...,  and  let  A  -  {a^}  be  a  skew  (k  x  k)  matrix,  then 
the  Pfafflan  of  A,  Pf(A),  Is  defined  as 

k  k  k 

Pf(A)  -  (2®al )-1  \  l  •••  V  K(j1fj2,...,jk) 

Jr1  v1  V1 

*  *i,)2  *  ‘33j4  *Wk  ’ 

It  is  well-known  that  [Pf (A) ] 2  »  dot  A. 

de  Bruijn  (1955)  has  established  the  following  expression  for  k  even. 

Leaaa  5.1.  Assuae  dot(  (4^(x^)})  e  H**)  and  let  k  •  2a  and  a  »  1,2,...,  then 

/  •••  /  det(U  (xi)})dx1dx2  •••  dxJc 
Q(a,b)  3  '  K 

b  b 

-  Pf({a1:j  -  /  /  ^  (*)4j  (y*sign(y  -  x)dxdy})  .  (5.4) 

Remark .  de  Bruijn  (1955)  gives  a  somewhat  unusual  definition  of  the  Pfafflan  and  his 

derivation  of  the  integral  on  the  left-hand  side  of  (5.4),  for  k  odd,  is  only  valid  in  a 
very  special  case.  However,  Krishnaiah  and  Chang  (1971,  equation  2.6)  give  a  general 
solution  to  the  odd  case.  In  their  notation  ♦^(x)  “  x1^3  %(x)  for  r  >  0  and  some 
function  t(x)  satisfying  the  integrability  conditions.  Me  restate  their  results  as 
T earns  5.2  (an  alternative  proof  is  given  in  langeland I ( 1960) ). 

leaaa  5.2.  Assuae  det(  {♦^(x^ ) } )  e  L(B^)  and  l«it  k  be  odd,  then 

k  ' 

/  •••  /  det({*.(x  )})dx  dx  •••  dx  «■  l  (-1)3  >  (b)Pf(A  ) 

Q(a,b)  3  *  :-1  3  3  ' 

where 

b 

♦j(b)  -  /  fjUJdt  for  j  -  1,2,. ...It  , 


and 


(5.5) 


f(x)  -  (2lt/2  n  rtj/2)]'1 

3-1 


Pf 


2G1,0<x) 


2Glc-t,o(*) 


2G0, 1(xi 


2G1,2<X> 


2G(J,k-1tx) 

0 


2Gk-2, 1<x>  0 

0  2G,c.1,2«x> 


2Gk-2,k-1(x> 

0 


Let  k  -  2a,  then,  according  to  definition  of  the  Pfaffian  and  the  relation  for  signature 
functions 

k  k  k 

K<xvx2 . xk)  -  <2*al>*1  l  l  •••  l  K(j1,j2 . 3k) 

3 1-1  3 2-1  3k-1 

*  *<x3,x32)  *  B<x33x34>  “^n-iV 

aatablishad  in  da  Bruijn  '1955),  thm  Pfaffian  in  (5.5)  can  ba  raducad  to 

a  a  a 

2*  I  I  •••  I  W3v32»-**r3«)  * 

3 i“i  3a-1  3,-1 

*  G0,2ji-1<x,'G2,2j2-1{x,*,,,'Gk-2,23,-t(x)  * 

But  this  is  nothing  but  2*  tiaes  the  deterainant  in  Theorea  2.1.  The  pr  of  is  coaiplete. 
Proof  of  Theorea  2.2 

k  _  k-1 

F<x)  -  (2k/2  I  r(i/2>]“  I  (-U^GWxJPfUa) 

i-1  J-O 

where  Aj  -  {a^}  le  «  (k  -  1)  *  (k  -  1)  aatrix  with  entries  a^  -  Gpfq  -  ®q,p  *or 
p,g  -  0,1,...  ,3  -  1,3  ♦  1, . .  •  ,k  -  1.  Next,  by  (2.7) 
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for  j  odd.  (It  can  also  be  shown  that  Pf(Aj)  «  0  for  j  odd.)  According  to  (2.9), 


for  j  even,  A^ 


G0,1<x> 


G1,2(x) 


G0,3+1(x) 


A3  -  2 


(k-i)  Gj-i.o<*> 

Gj+1,0(x) 


1 , 2<  x> 
c3+1,2(x) 


Gk-2,0<x) 


Gk-1,t(x> 


Gk-2,2(x) 


<**-1,  j-1lx) 


G0, j+1(x) 


G0,k-2(x) 


G0,k-1lx) 


Gj-1,k-1lx) 

Gj+1,k-1(x) 


Gk'-l,J+l{x> 


Gk-1,k-2<x* 


Gk-2,k-l‘x> 


All  entries  containing  J  a#  a  first  or  as  a  second  index  vanish,  t.e.,  all  G.  4(x)  and 

‘i; 

G  ,(x>  for  i  ■  1,3, ...,k  -  2  vanish,  the  retaining  n unbar  of  terns  G_  _(x),  with 
3  »*  PtH 

p  even,  is  exactly  (k  -  1)/2.  Thus,  the  Pfaffian  of  Aj  reduces  to 


at  m  m 

2(k  1)/2  ^  £  ...  ^  E( il»32',*"'3»*G0/2i<-1*x*  * 

ir1  V1  v1  1 


*  G2,2j2-1(x) 


Gi~2’2i{ j-2)/2“1<X)  *  J3+2'2i( j+2)/2"1<X) 


Gk-1,2j1>-1^>c* 


where  m  »  (k  -  1)/2.  Except  for  e  possible  sign  change  this  is  nothing  but  2^“’ ^2 
times  the  determinant  of  the  matrix  B(j/2)  appearing  in  the  statement  of  Theorem  2.2. 
By  inspection,  the  sign  is  given  by  (-1 ) <k-1)/2+j.  proof  is  complete. 

He  remark  that  nuclear  physicists  (e.g.  Mehta  (1967),  Wigner  (1967))  are  interested 
in  distributions  of  the  eigenvalues  of  S. 
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